MEMORANDUM February 5, 2014 
TO: Board Members 


FROM: Terry B. Grier, Ed.D. 
Superintendent of Schools 


SUBJECT: 2013 EVERYDAY EXCELLENCE INSTITUTE EVALUATION REPORT 
CONTACT: Carla Stevens, 713-556-6700 


Secondary-level English language learners (ELLs) are at risk of falling behind 
academically because of unaddressed gaps in academic language and literacy skills. 
The Everyday ExcELLence Institute was a collaboration between the Multilingual, 
Curriculum, Instruction, & Assessment, and Professional Support & Development 
departments and was intended to provide secondary-level ELL teachers with practical 
instructional routines that could be used in a variety of content areas. 


The report summarizes data from the ExcELLence Institute training for teachers which 
occurred in 2012-2013. Included are demographic data for program participants, 
information on teacher reactions to the training and on their implementation of the 
strategies they learned, as well as data on the impact of training on the academic 
performance of students of those teachers. 


A total of 493 teachers attended the Everyday ExcELLence Institute, teaching in the areas 
of reading/ELA, mathematics, science, or social studies. Results showed that teachers 
were satisfied overall with the quality of the training. Teachers reported using most 
ExcELLence Institute strategies fairly frequently, but did express concern over the amount 
of ongoing support they had available, particularly from principals and others 
administrators. Finally, performance of ESL students whose teachers received training 
showed some evidence for beneficial effects compared to ESL students whose teachers 
did not receive training. These effects depended on the teacher's core area and on the 
subject area tested, and teachers who implemented the strategies well appeared to have 


a greater impact on their students. 
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EVERYDAY EXCELLENCE INSTITUTE PROGRAM EVALUATION REPORT 
2012-2013 


Executive Summary 
Program Description 


More than 60,000 students in Houston ISD are labeled as “English language learners”, or ELLs. Many of 
these students have unaddressed gaps in academic language and literacy skills, particularly at the sec- 
ondary level. Without proper instructional supports, these students are at risk of falling behind academi- 
cally. The Everyday ExcELLence Institute was a four-day training session for teachers of secondary- 
level ELL students, and was held in August of 2012. The Institute was the product of a joint collaboration 
between the Multilingual, Curriculum, Instruction, & Assessment, and Professional Support and Devel- 
opment (PSD) departments of HISD, and was aimed at teachers in grades six through twelve. The intent 
of the Everyday ExcELLence Institute was to provide teachers with practical instructional routines that 
could be used with ELL students in any content area. 


Teachers attending the Institute were exposed to skills and practices that should allow them to better 
reach and engage their ELL students. These practices fell into two main categories. First, participants at 
the ExcELLence institute received training on sheltered instruction from ELL expert John Seidlitz. Shel- 
tered instruction training promotes and enhances the use of instructional strategies and modifications 
that allow ELLs to access an English language curriculum more effectively. Dr. Seidlitz’s practical ap- 
proach to "sheltering" English language learners emphasizes giving students the support they need to 
learn difficult new content while learning academic language. In addition, teachers were instructed in the 
use of eight literacy routines. Adolescent literacy and language acquisition research suggests that sim- 
ple, high-impact instructional actions can help ELLs learn more new content while developing stronger 
vocabulary and literacy skills. The everyday excellence routines were intended to be used daily, and are 


summarized on the PSD website at http://houstonisdpsd.org/literacy-routines.html. 


The sheltered instruction portion of the training was delivered over the first two days, while the eight liter- 
acy routines were reviewed on the final two days. Training was provided by a team of eighty district staff 
(Multilingual department staff and Teacher Development Specialists) who had themselves been trained 
on the various techniques covered. 


Highlights 


e A total of 493 teachers participated in the Everyday ExcELLence Institute, with 318 of them partici- 
pating for the full four days and an additional 128 attending for at least two days. 


e There was no statistically significant difference between Institute participants and other HISD teach- 
ers (Secondary) in either age or in amount of overall teaching experience. 


e The amount of HISD teaching experience did differ between the groups, with the Institute attendees 
being slightly less likely to have one or fewer years experience teaching in the district. 


e One hundred and three participants completed an online survey regarding their reactions to the 
training sessions, and their use of strategies learned while attending. 


e Overall, there was a high degree of satisfaction with the training, with 92% of teacher responses 
being positive. 


e Teachers were relatively less positive when asked to comment on their use of specific strategies 
during the school year. Some strategies were more difficult to use than others, and teachers also 
expressed concern over the support they received from administrators. 


e Performance of ESL students whose teachers received ExcELLence Institute training showed some 
evidence for beneficial effects, compared to those of ESL students whose teachers had not received 
similar training. 


e« These beneficial effects depended on both teachers’ core area (reading and mathematics teachers 
had a greater impact than did science or social studies teachers) and on the subject area tested. 


e« English language proficiency, as measured by the TELPAS, did show benefits for students of trained 
teachers, but only for students whose teacher's core area was reading. 


e Gains on TELPAS were significantly greater for students whose teachers had been identified as 
having good implementation of the strategies in their classrooms. 


Recommendations 


1. There is only sparse data concerning whether learned strategies are actually being implemented in 
the classroom. Teacher self-report via surveys has been relied on to provide some idea of what is 
happening, but a full analysis of program impact will need to find a better way of linking observed 
frequency of strategy use with actual student performance. 


2. Future implementations of this and similar programs should address the trends observed here, spe- 
cifically the weak benefits observed for teachers in the core areas of science and social studies. 
Training procedures may need to be reviewed. 


3. Responses on the teacher survey indicate that a lack of support for teachers by campus administra- 
tion and central office staff may be a problem. This issue has also come up in evaluations of previ- 
ous versions of this program. The Professional Support & Development (PSD) and Multilingual de- 
partments should identify ways in which campus administrators and Teacher Development Special- 
ists can support teachers who have gone through this training. 


Administrative Response 


Spring 2014 Walkthroughs: In order to further refine the model, PSD and Multilingual staff will conduct 
observations in Spring 2014 of classrooms with ExcELLence-trained teachers who have received robust 
coaching by Seidlitz-trained TDS. 


Administrative Support: PSD and Multilingual will provide more overview training and resources to cam- 
pus and central office administrators, including sessions at the Summer Leadership Institute and ses- 
sions offered to new and aspiring campus administrators. This support will emphasize the role of the 
English Language Proficiency Standards (ELPS, TAC 8 74.4), differentiating based on language profi- 
ciency levels, and implementation of the literacy routines. 


Additional Training: The Everyday ExcELLence institute will again be offered in summer 2014. The Mul- 
tilingual Department and PSD will continue to improve excELLence training and coaching based on data 
and stakeholder feedback. Additional training and an “endorsement” will be developed for teachers who 
have completed initial training and have actively implemented the Everyday ExcELLence routines. Early 
adopters will also be highlighted in upcoming training and promotional materials. PSD and Multilingual 
staff will also develop a Level 2 Everyday ExcELLence institute for the 2014-15 school year. 


Additional Capacity Building: TDS and Multilingual Specialists will continue to receive training, practice, 
and coaching to increase their capacity to train and coach teachers in routine implementation. 


Campus-Based Training/Coaching: As a critical adjunct to institute training, TDS will continue to deliver 
training and coaching in the distributed practice model on high-needs campuses (training one routine at 
a time then following up with targeted coaching and learning walks with administrators). 


Support Materials: PSD is creating video exemplars for the routines. Additional online and print re- 
sources will be created to increase awareness and circulation of the materials as well as to increase the 
number of teachers and administrators who access these materials to reinforce training concepts. 


School Administrator Input: School administrators will be surveyed during the annual Summer Leader- 
ship Institute to determine excELLence implementation plans and to select campuses which are commit- 
ted to providing additional training with ongoing follow-up and support during the school year. 


Introduction 


More than 60,000 students in Houston ISD are labeled as “English language learners”, or ELLs. Many of 
these students have unaddressed gaps in academic language and literacy skills. Without proper instruc- 
tional supports, these students are at risk of falling behind academically. The Everyday ExcELLence 
Institute, a four-day training session for teachers of ELL students, was held in August of 2012. The Insti- 
tute was the product of a collaboration between the Multilingual, Curriculum, Instruction & Assessment, 
and Professional Support and Development (PSD) departments of HISD, and was aimed at teachers in 
grades three through twelve. The intent of the Everyday ExcELLence Institute was to provide teachers 
with practical instructional routines that could be used with ELL students in any content area. 


Teachers attending the Institute were exposed to skills and practices that should allow them to better 
reach and engage their ELL students. These practices fell into two main categories. First, participants at 
the ExcELLence institute received training on sheltered instruction from ELL expert John Seidlitz. Shel- 
tered instruction training promotes and enhances the use of instructional strategies and modifications 
that allow ELLs to access an English language curriculum more effectively. Dr. Seidlitz’s practical ap- 
proach to "sheltering" English language learners emphasizes giving students the support they need to 
learn difficult new content while learning academic language. In addition, teachers were instructed in the 
use of eight literacy routines. Adolescent literacy and language acquisition research suggests that sim- 
ple, high-impact instructional actions can help ELLs learn more new content while developing stronger 
vocabulary and literacy skills. These eight routines were intended to be used daily, and are summarized 


on the PSD website at http://houstonisdpsd.org/literacy-routines.html. 


The sheltered instruction portion of the training was delivered over the first two days, while the eight liter- 
acy routines were reviewed on the final two days. Training was provided by a team of eighty district staff 
(Multilingual department staff and Teacher Development Specialists) who had themselves been trained 
on the various techniques covered. Appendix A (see p. 14) summarizes the eight everyday excellence 
routines, and Appendix B (see p. 16) provides further background on sheltered instruction. 


Methods 


Participants 


A total of 493 teachers attended the Everyday ExcELLence Institute in 2012—2013. Most of these (318 
teachers, or 65%) attended four full days, with an additional 128 (26% of teachers) attending at least two 
days and 47 (9.5%) attending only one day. All teachers received ongoing consultation with the teacher 
development specialists. Appendix C (See p. 18) shows counts of teaching and non-teaching staff who 
attended training by campus. Student performance data were analyzed from all ESL students who were 
in classes taught by teachers who attended the Everyday ExcELLence Institute. Data for all other ESL 
students in the district served as a comparison. 


Data Collection & Analysis 


The Multilingual Department provided of a list of teachers attending the Everyday ExcELLence Institute. 
Teacher’s employee ID codes were retrieved from the district's Chancery database in order to gather a 
list of classes which they taught. Next, teacher demographic information was extracted from Chancery, 
including years of teaching experience. A list was created of all students in classes taught by teachers 
who attended the training, which was then used to retrieve student performance data on various stand- 
ardized tests. Course grade results were also included in the analyses. 


An online survey was used to collect data from teachers and other staff who attended the Everyday Ex- 
cELLence Institute 2012—2013. The first section of the survey sought feedback from the institute at- 
tendees on their reactions to the training, what their experiences had been, what had worked, and what 
had not. A copy of the full survey, along with responses, is shown in Appendix D (p. 19). The second 
part of the survey concerned implementation of the various strategies they had learned. This survey in- 
cluded questions about implementing these methods in the classroom, as well as questions concerning 
teachers’ use of specific strategies (Appendices E-G, pp. 20-22). Teachers completed the survey online 
at the end of the school year. Appendices D through G also include a summary of responses. 


Student performance data were collected from the State of Texas Assessments of Academic Readiness 
(STAAR and STAAR End-of-Course), the Texas Assessment of Knowledge and Skills (TAKS), Stanford 
Achievement Test (Stanford 10), and the Texas English Language Proficiency Assessment System 
(TELPAS). STAAR results are reported for the reading and mathematics tests. For each test, the per- 
centage of students who met the Phase-In 1 Satisfactory standard is shown. For STAAR EOC, the per- 
cent of students who met the Satisfactory standard are reported for English | and I] Reading and Writing, 
Algebra |, Biology, Chemistry, Geometry, World Geography, and World History. For TAKS, the percent 
of students meeting standard are reported for the reading and mathematics tests. Stanford 10 results 
are reported (Normal Curve Equivalents or NCEs) for reading, mathematics, and language. 


TELPAS results are reported for two indicators. One indicator reflects attainment, i.e., the overall level of 
English language proficiency exhibited by ELL students. For this indicator, the percent of students at 
each proficiency level is presented. The second indicator reflects progress, i.e., whether students gained 
one or more levels of English language proficiency between testing in 2012 and 2013. For this second 
TELPAS indicator, the percent gaining one or more proficiency levels in the previous year is reported. 
Appendix H (see p. 23) provides further details on each of the assessments analyzed for this report. 


Student grades were included for selected courses that were aimed primarily at ELL students. These 
included courses in the core areas of reading/language arts, mathematics, science, and social studies. 


Results 
What was the demographic profile of teachers who received ExcELLence Institute training? 


e Figure 1 shows the distribution of ages for teachers who received ExcELLence Institute training 
(shaded bars). Also included, for comparison, is the relative distribution of ages for teachers in the 
district (open circles). Note that HISD data only includes middle and high school teachers. 


Figure 1. Distribution of Everyday ExcELLence Institute trained teachers by age. 
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Seventy-five percent of teachers receiving training were female and 25 percent were male. 

The mean age of teachers receiving training was 42.7 years (median = 42 years). 

The distributions of ExcELLence Institute teachers and other non-trained secondary-level teachers 
were roughly the same. The odds of being younger than 35 years old were almost the same for 
teachers who attended training as they were for other teachers in the district (odds ratio = 0.88, z = 


1.28, p > .10, not significant at p < .05). 


Figure 2. Distribution of trained teachers by years of previous experience teaching overall. 
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The average amount of prior teaching experience for trained teachers was 10.7 years (median = 6 
years). 


Figure 2 shows the distribution of prior experience (ExcELLence Institute teachers as bars, HISD 
overall as open circles). Trained teachers were no more likely to have one or fewer years of experi- 
ence than other teachers in the district (odds ratio = 0.86, z = 1.21, p>.11, not significant at p < .05). 


Figure 3. Distribution of trained teachers by years of previous experience teaching in HISD. 
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Figure 3 shows the distribution of teaching experience within HISD. ExcELLence Institute teachers 
were less likely to have one or fewer years of experience than other teachers in the district (odds 
ratio = 0.74, Zz = 2.66, p<.01, significant at p < .05). 


How Satisfied were teachers with the training received at the Everyday ExcELLence Institute? 


One hundred and three individuals who had attended ExcELLence Institute training responded to an 
online survey assessing reaction to the training sessions (this was 21% of all those who attended). A full 
summary of responses to the entire survey can be found in Appendix D (p. 19). 


e Respondents were equally divided between middle and high school teachers (49.0% each; with two 
non-teachers). Of the teacher respondents, 49.5% taught reading or English language arts, 26.2% 
taught math, 21.4% taught science, 18.4% taught social studies, and 17.5% taught other subects. 
Note that the total is greater than 100% because some taught multiple subjects. 


e Overall, responses to this survey indicated a high degree of satisfaction with the training, with 92% 
of responses being positive. 


e The trainers: Opinions about the trainers were highly positive, with 92 percent or more either agree- 
ing or strongly agreeing with the statements such as the following: “adequately set the tone and 
background for information presented” (95%), “actively encouraged collaborative discussion” (95%), 
and “allowed me to reflect and share my ideas/views about the topics presented” (94%). 


e The training sessions: Statements which received the highest degree of support were the following: 
“the learning outcomes for the sessions were clearly communicated” (96%), “the information was 
conveyed in a way that was easy to comprehend and follow” (93%), “the session(s) was/were rele- 
vant to my teaching/work within the school” (93%), and “the information was relevant and useful to 
my daily teaching/work” (92%). 


e The question with the lowest level of agreement was “I feel comfortable enough with the information 
| learned that | could share it with my colleagues,” with 86% either agreeing or strongly agreeing. 


How effectively were strategies implemented by teachers who attended the Institute? 


The effectiveness of implementation of ExcELLence Institute strategies was assessed via an online sur- 
vey completed by teachers who had attended the training sessions - this was a continuation of the sur- 
vey described above. The first six items in the survey concerned degree of difficulty faced when trying to 
implement the learned strategies in their classrooms (See Appendix E, p. 20). 


e Ease of implementation: In comparison with the questions concerning the reactions to the original 
training they received (See above), attitudes toward implementation of ExcELLence Institute strate- 
gies were less positive. 


« The most positive responses were to the item “things | learned during training were easily imple- 
mented in the classroom” (85%). Teachers also reported observing positive benefits for students 
after using the strategies in their classroom (83%), and felt that students liked the inclusion of them 
in their classes (81%). 


e Positive reaction fell off quickly after this, particularly to survey items that concerned support or as- 
sistance they had received; “other district staff facilitated my use of these strategies” (45% agree- 
ment), and “principals and other administrators facilitated my use of these strategies” (30%). 


e« Nearly half of teachers (48%) felt that including the ExcELLence Institute strategies in their teaching 
increased their workload. 


- How often were specific strategies used: Seventeen items in the survey asked how frequently teach- 
ers used specific stragies in their classroom during the year (See Appendix F for responses, p. 21). 


e Among the most frequently used strategies were: "randomize/rotate to call on students" (85% 
"usually" or “always"), "model and use complete sentences" (83%), "post and use word walls" and 
"implement language and content objectives" (77%), "use stems to develop language and academic 
vocabulary" (74%), "scaffolding using a gradual release model" (71%), and "use of response sig- 
nals" (70%). 


e Methods used less frequently included "Use of Huddle" (42%), "Be the Lead Reader" (48%), and 
"Turn the Light On" (49%). 


« How easy was it to use specific strategies: Seventeen items also asked how easy or difficult it was 
to use the various learned strategies in their classroom (See Appendix G, p. 22). 


e Not surprisingly, whether a strategy was judged to be "very easy" or "easy" to use was related to 
how frequently it was used. The correlation between these two variables was significant (r = 0.71, p 
< .01, statistically significant at p < .05). 


e Strategies judged to be the easiest to use were "use of response signals" (81% "very easy” or 
"easy) and "randomize/rotate to call on students" (81%). Those judged to be more difficult included 
"Use of Huddle" (55%) and "Be the Lead Reader" (58%). 


What was the impact of ExcELLence Institute training on the academic performance of students 
in classes taught by trained teachers? 


A detailed explanation of data analysis procedures for student performance can be found in Appendix | 
(p. 24). Table 1 provides a brief summary of the various student performance measures which were an- 
alyzed. Only data from secondary students were analyzed. Briefly, student performance data included 


Table 1. Student Performance Matrix, Including the Subject Tests Included for Each Assessment 


Course 
Tested Subject 3-8 EOC TAKS Stanford TELPAS Grades 
Math 1 2 (elgebre 1 1 1 
geometr 
Reading 1 2 (reading 1 1 2 (reading, yearly 1 
1 & ll) progress) 
, 2 (biology, 
Science 1 chemistry) 1 1 1 
2 (world 
Social Studies 1 history & 1 1 1 
geography) 
Language/Writing . ae 1 
Scale Score 
Scale Scale Scale (Reading) & Final Grade 
Performance Measure Score Score Score Ne Percent Gained Avg 
(Yearly Progress) 


results from five different standardized tests as well as course grades in four core areas. For example, 
the STAAR 3-8 assessment included tests for mathematics, reading, science, and social studies. The 
performance measure analyzed for the STAAR 3-8 was the scale score on each of these four tests. For 
the STAAR EOC, ten separate test results were analyzed; two each for mathematics (algebra and ge- 
ometry), reading (English | & II Reading), science (biology & chemistry), social studies (world history and 
world geography), and writing (English | & Il Writing). TAKS and Stanford 10 can be interpreted similarly. 


For the TELPAS, results were analyzed for reading (reading scale score) and yearly progress (percent 
of student who made gains in proficiency between 2012 and 2013). For student grade data, results were 
averaged across courses, so that (for example) a student received only one grade for mathemartics 
even if they took multiple mathematics courses. Only the final grade average was included for each 
course a student may have taken. A list of courses included is shown in Appendix J (p. 25) 


Detailed student performance results can be found in Appendix K (pp. 26-27). Summary results are 
shown schematically in Table 2, which should be interpreted as follows: 


e On the left side of the matrix are the five subject areas tested; mathematics, reading, science, social 
studies, and writing/language arts. 


e At the top of the matrix are the four self-identified subject areas taught by teachers who participated 
in the ExcELLence Institute training. Each teacher selected one and only one of these core areas. 


e All results are summed across the various different assessments shown in Table 2 (STAAR, EOC, 
Stanford, etc.). 


e Each cell shows the number of different analyses which showed a significant performance ad- 
vantage for students whose teachers participated in training, compared to those whose teachers 
had never received any similar training. 


e For example, the cell representing the intersection of reading/ELA and reading (dark green) shows 
that out of the eight different measures analyzed, three showed a significant performance advantage 
on reading assessments for ESL students of trained teachers whose core area was reading. 


e Similarly, the intersection of the reading and science columns (highlighted in red) showed that stu- 
dents whose science teachers received training did not show a performance advantage in reading 
for any of the eight different measures analyzed. 


e Overall, students whose mathematics or reading/ELA teachers were trained showed the most per- 
formance gains, with all subject areas showing some evidence for improvement. 


Table 2. Schematic Summary of Student Performance Results (see Appendix K for details) 


Teacher's Trained Subject 


Math Read/ELA Science SocStudies 


Math 
Reading 


Tested 


Subject Science 
Social Studies 


Language/Writing 


Figure 4. TELPAS results for Excellence Institute and other ESL students 2013: A. Adjusted scale 
score in reading, and B. Percent making gains in English proficiency 
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Source: TELPAS, Chancery 


e In total, 8 out of 29 separate analyses showed performance benefits for students whose reading 
teachers were trained (> 27%), compared to 7 out of 29 for mathematics teachers (> 24%, two left- 
most columns in Table 2). 


e In contrast, students whose science teachers participated in the ExcELLence Institute showed only 
weak evidence for performance gains, with only 1 out of 29 separate analyses (< 3%) showing a 
significant advantage compared to students of untrained teachers (third column in Table 2). 


e Interms of subject area, language/writing assessments showed the most evidence for performance 
gains, with 4 of 12 measures analyzed showing significant benefits for students of trained teachers 
(fifth row of Table 2). 


e Reading (6 of 32) and science (6 of 24) also showed some evidence for performance benefits from 
ExcELLence Institute training, with mathematics (2 of 24) showing the weakest evidence. 


e« In summary, teachers of science, and possibly social studies, showed less of a benefit from the Ex- 
cELLence Institute training, with mathematics and social studies assessments showing less im- 
provement than other subject areas. 


e Since TELPAS performance is a key metric used to assess performance of ELL students, Figure 4 
shows results for the TELPAS reading and TELPAS yearly progress measures. 


e TELPAS results showed that students of trained teachers had a higher average reading scale score, 
and that a higher percentage of them showed progress in English proficiancy, than did students of 
untrained teachers. However, this was only true for reading/ELA teachers; teachers of other core 
subjects did not improve their students TELPAS performance if trained. 


What was the impact of ExcELLence Institute training on the academic performance of students 
whose teachers were known to have implemented the strategies? 


The data overall show that there are performance benefits for students of trained teachers, but that 


these effects are modest, and dependent on both the core subject taught by the teacher and the subject 
tested. An important question is whether these benefits can be shown to be greater for teachers who are 


10 


known to have implemented and used the strategies in their classrooms. The final data analysis report- 
ed here summarizes results from just this type of analysis. In order to address this issue we took ad- 
vantage of a follow-up training session that was held in May of 2013. 


Institute attendees were invited to deliver presentations to showcase implementation of the Everyday 
ExcELLence routines at a training event in May 2013, the Everyday ExcELLence Expo. Interested 
teachers submitted a proposal form that included the teacher’s campus, grade level, content area, rou- 
tine(s), and a description of the practices to be presented. These proposals were reviewed by a commit- 
tee of Teacher Development Specialists (TDS) and Multilingual Program (MPS) Specialists. The pro- 
posal review process, as well as “presentation coaching” for all selected presenters, resulted in high 
quality presentations. 


Expo presenters tended to be eager institute attendees or teachers who had been highly coached by 
their TDSs throughout the year. Quantitative data regarding frequency of literacy routines coaching ses- 
sions is not available, so the “highly coached” identification is anecdotal and based on TDS and MLP 
specialists’ reports during Everyday ExcELLence Expo steering meetings. Specialists who agreed to 
serve as “presentation coaches” followed checkpoints established by project leadership and shared the 
presenters’ progress during regularly scheduled meetings prior to the Expo. 


The steering committee selected 20 teachers to present at the Everyday ExcELLence Expo training 
event. Of these, seven teachers were selected due to participant anecdotal reporting of session rele- 
vance and quality. The data from these seven teachers was reviewed to determine if high implementa- 
tion of the routines would show a positive academic impact for ELLs. /t is important to note that these 
seven teachers were identified not by a review of student performance data, but on the basis of observ- 
ers judgements as to how effectively teachers were using the strategies in their classrooms. 


« Summary data for TELPAS is shown in Figure 5. Data from students of the targeted teachers is 
shown in red, data from students of all other trained teachers is in blue, and data from ESL students 
whose teachers received no training is shown in yellow. 


« Students of targeted teachers showed large benefits on both overall TELPAS reading (46 scale 
score points) and in the percentage of students who improved their TELPAS rating in 2013 (28 per- 
centage points), compared to students of all other trained teachers. 


Figure 5. TELPAS results for students of targeted teachers, non-targeted teachers, and other 
ESL students, 2013: A. Adjusted scale score in reading, and B. Percent making gains in English 
proficiency 
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Discussion 


The goal of the Everyday ExcELLence Institute training was to provide secondary ELL teachers with 
practical instructional routines that could be used with ELL students in any content area. This included 
an overview of sheltered instruction techniques, as well as eight other literacy routines. Data suggest 
that even though those teachers who responded to a Survey indicated that they used the various ExcEL- 
Lence Institute strategies to at least some extent in their teaching, the impact of training on student per- 
formance was highly variable. Teachers from some core areas (reading and mathematics) showed evi- 
dence for beneficial effects on their students, while those in other areas (e.g., science) showed little evi- 
dence of benefits. Overall, reading teachers seemed to have the strongest evidence for improved stu- 
dent performance, and reading/writing also was the subject area or skill that seemed to have benefited 
the most from teacher participation in the institute. English language proficiency, as measured by the 
TELPAS, was positively affected by teacher participation in the ExcELLence Institute, but only if the 
teacher's core area was reading. However, students showed large gains in English proficiency if their 
teachers were judged to have implemented the strategies to a high degree. 


Evaluations of two prior versions of sheltered instruction training offered by the district (Houston Inde- 
pendent School District, 2010; 2011) found evidence for small but beneficial effects of sheltered instruc- 
tion training on student performance. A similar evaluation conducted in 2012 (Houston Independent 
School District, 2012) found no consistent benefits for students of teachers who received a combination 
of sheltered instruction and literacy development training. The present findings suggest that further in- 
vestigation is needed, and also that the training might need to be revised for teachers of certain core 
subject areas. 


Limitations 


This study assumed that a student whose teacher went through ExcELLence Institute training were ex- 
posed to the “treatment” (i.e., use of certain strategies in classes they take), and that the benefits of 
teacher participation would be reflected in student performance on STAAR and other assessments. 
However, students take a variety of classes with different teachers at the secondary level, and students 
also differ greatly in terms of a whole variety of demographic and personal factors, any of which may 
mask beneficial effects of the “treatment”. It is also true that benefits from exposure to ExcELLence Insti- 
tute strategies in the classroom may occur only over the long-term, or following prolonged exposure to 
such techniques. Neither of these was possible given the brief timeline involved. Another limitation of the 
report is that there was little data available on how well the strategies were actually being implemented 
in the classroom by teachers. Such data, if reliable, could provide for a more powerful analysis of pro- 
gram impact since student performance benefits should be related to whether teachers did actually use 
the strategies effectively. Present data do not allow this to be done, however. 
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Appendix B 
Sheltered Instruction Background Information 


Sheltered instruction is a style of teaching which makes grade-level academic content in core areas 
(e.g., math, science, social studies) more accessible for English Language Learners (ELLs), while at the 
same time promoting development of English language proficiency. It highlights key language features 
and incorporates strategies to make content more comprehensible to students, without sacrificing rigor. 
Sheltered instruction is sometimes referred to as SDAIE (specially designed academic instruction in 
English). While use of sheltered instruction techniques has come to be widespread in U.S. schools, this 
growth has often been characterized by inconsistent practices from district to district, and even from 
class to class within the same school (August & Hakuta, 1997; Berman et al, 1994; Kaufman, et al., 
1994; Sheppard, 1995; Short, 1998) 


Arguably, the most popular version currently in use is the sheltered instruction observational protocol, or 
SIOP (Echevarria & Graves, 1998; Echevarria, Vogt, & Short, 2000). The SIOP model was developed in 
a seven-year national research project (1996-2003) sponsored by the Center for Research on Educa- 
tion, Diversity, and Excellence (CREDE). Researchers identified features of instruction present in high- 
quality sheltered lessons, and developed an observational tool consisting of 30 items grouped into three 
sections: preparation, instruction, and review/evaluation. The instruction component is further broken 
down into clusters of items dealing with building background, comprehensible input, strategies, interac- 
tion, practice/application, and lesson delivery. 


All features of the SIOP model are aligned with current research on instruction for ELLs. SIOP was origi- 
nally designed to be used as an observation and rating tool for researchers, but it was soon recognized 
that the instrument could be used by teachers for lesson planning and reflection. Some of the tech- 
niques and strategies which SIOP encourages include the following: 


e use of supplemental materials, 

e adapt content to level of student proficiency, 

e link concepts to student background and experiences, 

e — link past learning and new concepts, 

e use scaffolding techniques, 

e allow for frequent interactions between student-teacher and among students, 

e use hands-on materials or manipulatives, and 

e provide activities that integrate all language skills (reading, writing, listening, speaking). 


Research has shown that the SIOP model is effective for learners at all grade levels across many sub- 
ject areas, and can impact student achievement (Echevarria, Vogt, & Short, 2004). 


District teachers received SIOP training in 2009-2010, and two different evaluations (Houston Independ- 
ent School District, 2010; 2011) found evidence of performance gains for students whose teachers had 
received sheltered instruction training, 


Sheltered instruction training for district ELL teachers has undergone modification in the past two years. 
A significant factor in this has been the prominence of literacy issues figure for the district's secondary 
ELL students. Results of the 2011 NAEP reading test showed that 18% of district students in grade 8 
were at least proficient in reading (i.e., reading at roughly an 8th grade level or higher). However, 
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Appendix B (continued) 


only 2% of 8th grade ELL students were rated as proficient (NCES, 2012). Poor reading skills constitute 
a significant barrier for ELL students. This is because addition to interfering with their ability to master 
course content, inadequate reading skills prevent many ELLs from exiting ELL status (ELLS must meet 
specific standards in reading, writing, and oral English proficiency in order to cease being classified as 
ELL). Accordingly, sheltered instruction training in 2011-2012 was augmented by including a review of 
various strategies meant to improve student literacy. This portion of training borrowed heavily from a 
recent review by Beers (2003). Subsequently, in 2012-2013, the ExcELLence Institute attempted an 
approach which simplified things by reducing the number of individual strategies that were taught to sec- 
ondary ELL teachers, while still placing a dual emphasis on sheltered instruction and strategies for im- 
proving student's literacy. 
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Appendix C 


Number of Teachers and Other Staff Attending the Everyday ExcELLence Institute 
in 2012-2013, by Campus 


# # # # 


Campus Teachers Others Total | Campus Teachers Others Total 
Attucks MS 2 0 2 Las Americas MS 5 0 5 
Austin HS 9 0 9 Lee HS 5 0 5 
Barbara Jordan HS 2 0 2 Liberty HS 6 0 6 
Bellaire HS 7 0 7 Long MS 5 0 5 
Black MS 8 0 8 Madison HS 15 1 16 
Briarmeadow MS 3 0 3 Marshall MS 4 0 4 
Burbank MS 10 0 10 McReynolds MS 9 1 10 
Chavez HS 17 0 17 Milby HS 17 0 17 
Clifton MS 13 0 13 N Q Henderson ES 2 0 2 
Community Services 3 0 3 ortiz MS 9 0 9 
Cullen MS 4 0 4 Pershing MS 6 0 6 
Davis HS 2 0 2 Pin Oak MS 13 0 13 
Deady MS 28 1 29 Reagan HS 9 0 9 
DeBakey HSHP 1 0 1 Reagan K-8 ul 0 1 
Dowling MS 5 0 5 Revere MS 6 0 6 
Eastwood Academy HS 4 0 4 Rice MS 1 0 1 
Edison MS 8 1 9 Sam Houston MSTC 17 0 17 
Empowerment CP HS 1 0 1 Scarborough HS 2 0 2 
Energ For Exc Acad MS 4 0 4 Sharpstown HS 3 0 3 
Fleming MS 13 1 14 Sharpstown International 11 0 11 
Fondren MS 4 0 4 Sterling HS 3 0 3 
Fonville MS 6 0 6 Stevenson MS 4 0 4 
Furr HS 1 0 1 Sugar Grove Academy 5 0 5 
Grady MS 1 0 1 Thomas MS 2 0 2 
Hamilton MS 11 2 13 Waltrip HS 3 0 3 
Harper Alternative 1 0 1 Washington BT HS 3 0 3 
Hartman MS 8 0 8 Welch MS 9 0 9 
Henry MS 16 0 16 West Briar MS 6 0 6 
Hogg MS 7 0 7 Westbury HS 42 3 45 
Holland MS 9 0 9 Westside HS 6 0 6 
HS Ahead Academy 3 0 3 Wheatley HS 8 1 9 
Jackson MS 8 0 8 Williams MS 10 0 10 
Johnston MS 9 0 9 Woodson K-8 2 0 2 
Jones HS 1 0 1 Worthing HS 1 0 1 
Kashmere HS 2 1 3 Worthing MS 1 0 1 
Key MS 2 0 2 Yates HS 2 0 2 
Lamar HS 2 0 2 Young Men's CollegePrep 1 0 1 
Lanier MS 3 0 3 Young Women's College Prep 1 0 1 
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Appendix D 


Questions and Responses From Online Survey Administered to 
Everyday ExcELLence Institute Participants 


Items concerning the trainers/facilitators: 


Strongly Strongly 
Survey Item Agree Agree Neutral Disagree Disagree 
Adequately set the tone and background for 58% 37% 3% 2% 0% 
information presented in the session(s) (59) (38) (3) (2) (0) 
Allowed me to reflect and share my ideas/views 54% 40% 5% 1% 0% 
about the topics presented (55) (41) (5) (1) (0) 
Helped me to make connections with the 58% 34% 7% 1% 0% 
information so that | could use it in my teaching (59) (35) (7) (1) (0) 
Actively encouraged collaborative discussion oie oe ae ie ue 
(61) (34) (4) (1) (0) 
Items concerning the sessions themselves: 
Strongly Strongly 
Survey Item Agree Agree Neutral Disagree Disagree 
The information was relevant and useful to my 54% 38% 7% 2% 0% 
daily teaching/work (54) (38) (7) (2) (0) 
The topics were well organized and paced sere so78 oe oe oO” 
(57) (33) (6) (3) (0) 
The learning outcomes for the sessions were 58% 38% 3% 1% 0% 
clearly communicated (58) (38) (3) (1) (0) 
| feel comfortable enough with the information | 46% 40% 13% 1% 1% 
learned that | could share it with my colleagues (46) (40) (13) (1) (1) 
Handouts were useful and adequately supported 57% 34% 8% 1% 0% 
the information presented (58) (34) (8) (1) (0) 
The information was conveyed in a way that was 57% 36% 5% 2% 0% 
easy to comprehend and follow (58) (36) (5) (2) (0) 
My awareness of these teaching strategies was 55% 35% 6% 3% 0% 
enhanced (55) (35) (6) (3) (0) 
| am prepared to use the strategies in my teaching se Se ae eu8 We 
(53) (36) (9) (2) (1) 
Overall, the session(s) was/were relevant to my 53% 40% 7% 1% 0% 
teaching/work within the school (53) (40) (7) (1) (0) 
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Appendix E 


Questions and Responses From Online Survey Administered to 
Everyday ExcELLence Institute Teachers Concerning the Overall Ease 
of Implementing the Strategies in Their Classroom 


How easy was it to use the methods you learned about in the classroom? 


Strongly Strongly 
Survey Item Agree Agree Neutral Disagree Disagree 
Things | learned during training were easily 41% 44% 11% 3% 1% 
implemented in the classroom (41) (44) (11) (3) (1) 
Including the strategies in my teaching increased 16% 32% 22% 26% A% 
my workload (16) (2) (22) (26) (4) 
| observed positive benefits for students after 42% 41% 15% 1% 1% 
using these strategies in my classroom (42) (41) (15) (1) (1) 
Students appear to like the inclusion of these 33% 48% 16% 1% 2% 
strategies in my classroom (32) (47) (16) (1) (2) 
eae ane tee 20% 25% 33% 15% NG 
eae: (20) = (25)— (33)_— (18) (6) 
Principals and other administrators facilitated my 11% 19% 41% 19% 10% 
use of these strategies (10) (17) (37) (17) (9) 
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Appendix F 


Questions and Responses From Online Survey Administered to Teachers 
Concerning How Often They Used Specific Strategies 


Survey Item Always Usually Sometimes Seldom Never 
Pump Up The Vocab pi oe a a - 
Let's Talk ea on oe BY oe 
Be The Lead Reader ne ie oe ae - 
Pen/cil to Paper oa a eo a A 
Turn The Light On ny i a es i 
Do | Really Get It? ms ro oe ae ate 
= a ae 
Get To Know Me ne ine an 8 se 
Implement Language and Content Objectives ) 8 eh Be rh 
Use Response Signals pp an rt on a 
Question Using Q3SA (Question, Signal, Stem, 21% 28% 32% 14% 6% 
Share, Assess) (19) (25) (28) (12) (5) 
Embed gestures Into Learning Process ye ye mts rae on 
0, 0, 0 0, 0, 
Post And Use Word Walls “2 me ri i : 2 
Model And Use Complete Sentences oes ae ret i a 
RandomizeAnd Rotate To Call On Students pee — ri i mY 
Use Sentence Stems To Develop Language 46% 28% 22% A% 0% 
And Academic Vocabulary (42) (26) (20) (4) (0) 
Scaffold Using A Gradual Release Model (e.g., 40% 31% 22% 8% 0% 
"| Do, We Do, You Do") (36) (28) (20) (7) (0) 
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Appendix G 


Questions and Responses From Online Survey Administered to Teachers 
Concerning How Easy or Difficult It Was to Use Specific Strategies 


Very Very 
Survey Item Easy Easy Neutral Difficult Difficult 

22% 43% 31% 2% 1% 

Pump Up The Vocab (19) (37) (27) (2) (1) 
: 24% 52% 22% 1% 1% 

Bele lars (20) (44) (19) (1) (1) 
22% 36% 35% 5% 2% 

Be The Lead Reader (19) (31) (30) (4) (2) 
F 26% 47% 24% 4% 0% 

Pen/cil to Paper (22) (40) (21) (3) (0) 
13% 49% 34% 2% 1% 

Turn The Light On (11) (42) (29) (2) (1) 
18% 55% 25% 1% 0% 

? 

Do | Really Get It? (15) (46) (21) (1) (0) 
19% 36% 35% 8% 2% 

a (16) (30) (29) 7) (2) 
22% 52% 22% 2% 1% 

Get To Know Me (19) (44) (19) (2) (1) 
+ ati 16% 50% 28% 6% 0% 

Implement Language and Content Objectives (14) (43) (24) (5) (0) 
‘ 33% 48% 14% 4% 1% 

Use Response Signals (28) (41) (12) (3) (1) 
Question Using Q3SA (Question, Signal, Stem, 18% A3% 25% 13% 1% 

Share, Assess) (16) (37) (22) (11) (1) 
F 31% 42% 23% 2% 1% 

Embed gestures Into Learning Process (27) (36) (20) (2) (1) 
39% 39% 14% 7% 1% 

Post And Use Word Walls (33) (33) (12) 6) (1) 
38% 42% 13% 7% 0% 

Model And Use Complete Sentences (33) (36) (11) (6) (0) 
42% 39% 15% 4% 0% 
RandomizeAnd Rotate To Call On Students (36) (33) (13) (3) (0) 
Use Sentence Stems To Develop Language 31% 46% 18% 5% 0% 
And Academic Vocabulary (27) (40) (16) (4) (02) 
Scaffold Using A Gradual Release Model (e.g., 24% 41% 31% 3% 0% 
"| Do, We Do, You Do") (21) (36) (27) (3) (0) 
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Appendix H 
Explanation of Assessments Included in Report 


The STAAR is a state-mandated, criterion-referenced assessment used to measure student achieve- 
ment. STAAR measures academic achievement in reading and mathematics in grades 3-8; writing at 
grades 4 and 7; social studies in grades 8; and science at grades 5 and 8. The STAARLL is a linguisti- 
cally accommodated version of the STAAR given to ELLs who meet certain eligibility requirements. 


For high school students, STAAR includes end-of-course (EOC) exams in English language arts 
(English I, Il, and III), mathematics (Algebra |, Geometry, Algebra Il), science (Biology, Chemistry, Phys- 
ics), and social studies (World Geography, World History, U.S. History). In 2011-2012, only grade 9 stu- 
dents took the EOC exams, while those in grades 10 and 11 continued to take the TAKS. 


The TAKS is a state-mandated, criterion-referenced test first administered in the spring of 2003, and 
which is being phased out beginning in 2012. It measures academic achievement in reading, mathemat- 
ics, science, and social studies in grades 10 and 11. Students currently in grades 10 and higher as of 
2011-2012 will continue to take exit-level TAKS tests in order to graduate, while those in grades 9 and 
lower will instead take STAAR EOC exams (see above). 


The Stanford 10 is a norm-referenced, standardized achievement test in English used to assess stu- 
dents’ level of content mastery. Stanford 10 tests exist for reading, mathematics, and language (grades 
1-8), science (3-8), and social science (grades 3-8). This test provides a means of determining the rel- 
ative standing of students’ academic performance when compared to the performance of students from 
a nationally-representative sample. 


The TELPAS is an English language proficiency assessment which is administered to all ELL students 
in kindergarten through twelfth grade, and which was developed by the Texas Education Agency (TEA) 
in response to federal testing requirements. Proficiency scores in the domains of listening, speaking, 
reading, and writing are used to calculate a composite score. Composite scores are in turn used to indi- 
cate where ELL students are on a continuum of English language development. This continuum, based 
on the stages of language development for second language learners, is divided into four proficiency 
levels: Beginning, Intermediate, Advanced, and Advanced High. 


Student final grades in selected courses were analyzed. The courses were in the areas of reading, Eng- 
lish language arts, mathematics, science, and social studies. The analyzed courses did not include all 
courses taken by ESL students in these areas, but a subset which were specifically targeted at ELL stu- 
dents. A full list of courses included in the analysis is shown in Appendix J (p. 25). 
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Appendix | 
Analysis of Student Performance Data 


Student performance data was analyzed for the STAAR, STAAR EOC, TAKS, Stanford 10, and 
TELPAS. Also analyzed were final course grades from selected courses for ELL students. The following 
describes some of factors that were important in these analyses: 


Post Hoc Design: The study used a post hoc design, where students were identified only after being 
taught by one of the teachers who received training. This meant that it was not possible to select an ap- 
propriately matched comparison group. Instead, the group of comparison students was composed of all 
other secondary ESL students whose teachers had not participated in the ExcELLence Institute. 


Analysis of Covariance: Since treatment and comparison groups could be precisely matched, all anal- 
yses used an analysis of covariance procedure. In this, the students performance in 2013 was analyzed, 
with their performance in 2012 serving as a covariate. In this way, student performance in 2013 was cor- 
rected to take into account their prior performance level. 


Teachers Who Were Previously Trained: A related difficulty concerned the fact that some version of 
sheltered instructional training for secondary ELL teachers has occurred in three previous years in the 
district. Therefore, it is not sufficient to simply compare student of teachers trained in 2012-2013 to stu- 
dents whose teachers were not trained in 2012—2013, since some of these "comparison" teachers may 
well have been trained in the use of similar strategies in previous years. Instead, students of teachers 
trained in the current year need to be compared to students of teachers who had never been trained. 
This was accomplished by recording, for each teacher who participated in the ExcELLence Institute, a 
record of how many previous training they had received, if any. 


Teacher Subject Area: Another important aspect of the analyses was that data were analyzed separate- 
ly for different groups of teachers who specialized by subject area. For example, results for mathematics 
teachers who were trained were analyzed separately from those of teachers who specialized in reading/ 
ELA. This was done because it made it easier to determine whether students benefited by having their 
teachers of specific subject areas trained in use of these ESL strategies. In addition, it potentially al- 
lowed differences between teachers across subject area to be seen, which could facilitate modifications 
to training procedures in the future. 


Covariates: For STAAR, TAKS, and TELPAS reading, the data analyzed were the scale scores for par- 
ticular subjects from 2013. The covariates used were the corresponding data for that student from 2012. 
If a student did not have two years of results then their data were excluded from analysis. For the 
STAAR EOC, scale scores from ten different subject tests in 2013 were analyzed, but since students 
seldom took the same EOC test in the previous year, their 2012 TELPAS reading scale score was used 
as the covariate. For the the Stanford 10, the NCE from 2013 was the dependent variable, with NCE 
from 2012 was the covariate. All assessment subtests or subjects were analyzed separately. 


Course Grades: Course grade data consisted of the final grade in the areas of reading/English language 
arts, mathematics, science, and social studies. Courses included were those which were identified by 
staff from the multilingual department as being specifically targeted at secondary ELL students. If a stu- 
dent took multiple courses within one area, the average grade across all courses was used. For all 
course grade analyses, the TELPAS reading scale score from 2012 served as the covariate. 
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Appendix J 


Courses Included in Grade Analyses for ESL Students 


Course Number Course Name Course Number Course Name Course Number Course Name 
ELA1508A ESL CREATIVE WRITING A | ENG1266A ENG SOL 2A-INT ESL MATH MDLS B 
ELA1508B ESL CREATIVE WRITING B ENG1266B ENG SOL 2B-INT MTH2359A ALGEBRA 2A-ESL 
ELA1518A ESL PRAC WRITING A ENG1268A ENG SOL 2A-ADV MTH2359B ALGEBRA 2B-ESL 
ELA1518B ESL PRAC WRITING B ENG1268B ENG SOL 2B-ADV RDG16062 ORAL LNG DEV 6 
ELA1528A ESL TECH WRITING A ENG1366A ENG 3A INT RDG16065 ESL READ 6-BEG 
ELA1528B ESL TECH WRITING B ENG1366B ENG 3B INT RDG16066 ESL READ 6-INT 
ELD1976A NEWCOMER ELD A ENG1368A ENG 3A ADV RDG16068 ESL READ 6-ADV 
ELD1977A NEWCOMER ELD B ENG1368B ENG 3B ADV RDG16069 TRANS READ 6 
ELD1979A ESOL SCIENCE 1A ENG1369A ENG 3A TRANS RDG16072 ORAL LNG DEV 7 
ELD1979B ESOL SCIENCE 1B ENG1369B ENG 3B TRANS RDG16075 ESL READ 7-BEG 
ENG 10690 ENG 6 TRANS ENG1466A ENG 4A INT RDG16076 ESL READ 7-INT 
ENG1073E ENG 7 PREAP ESL ENG1466B ENG 4B INT RDG16078 ESL READ 7-ADV 
ENG10790 ENGLISH 7 TRANS ENG1468A ENG 4A ADV RDG16079 TRANS READ 7 
ENG1083E ENG 8 PREAP ESL ENG1468B ENG 4B ADV RDG16085 ESL READ 8-BEG 
ENG10890 ENGLISH 8 TRANS ENG1469A ENG 4A TRANS RDG16086 ESL READ 8-INT 
ENG1153C ENG 1A PAP ESL ENG1469B ENG 4B TRANS RDG16088 ESL READ 8-ADV 
ENG1153D ENG 1B PAP ESL ESL10640 ESL 6 BEG(LEV1) RDG1718A ESL READ 1A 
ENG1158A ENG 1A ADV ESL10660 ESL 6 INT(LEV2) RDG1718B ESL READ 1B 
ENG1158B ENG 1B ADV ESL10680 ESL 6 ADV(LEV3) RDG1728A ESL READ 2A 
ENG1159A ENGLISH 1A TRAN ESL10740 ESL 7 BEG(LEV1) RDG1728B ESL READ 2B 
ENG1159B ENGLISH 1B TRAN ESL10760 ESL 7 INT(LEV2) RDG1738A ESL READ 3A 
ENG1162A ENG SOL 1A-PRE ESL10780 ESL 7 ADV(LEV3) RDG1738B ESL READ 3B 
ENG1162B ENG SOL 1B-PRE ESL10840 ESL 8 BEG(LEV1) SCI40690 ESL SCIENCE 6 
ENG1164A ENG SOL 1A-BG ESL10860 ESL 8 INT(LEV2) SC140790 ESL SCIENCE 7 
ENG1164B ENG SOL 1B-BG ESL10880 ESL 8 ADV(LEV3) SC140890 ESL SCIENCE 8 
ENG1166A ENG SOL 1A-INT LOC9004A FOUND MATH A SCI4259A BIOLOGY A-ESL 
ENG1166B ENG SOL 1B-INT LOC9004B FOUND MATH B SCI4259B BIOLOGY B-ESL 
ENG1168A ENG SOL 1A-ADV MTH20690 MATH 6-ESL SST30690 ESL W CULT STDY6 
ENG1168B ENG SOL 1B-ADV MTH20790 MATH 7-ESL SST30790 ESL TEX HIST 7 
ENG1253C ENG 2A PAP ESL MTH20890 MATH 8-ESL SST30890 ESL US HIST 8 
ENG1253D ENG 2B PAP ESL MTH2159A ALGEBRA 1A-ESL SST3159A ESL W GEO STDYA 
ENG1258A ENG 2A ADV MTH2159B ALGEBRA 1B-ESL SST3159B ESL W GEO STDYB 
ENG1258B ENG 2B ADV MTH2259A ESL GEOMETRY A SST3259A ESL W HISTORY A 
ENG1259A ENG 2A TRANS MTH2259B ESL GEOMETRY B SST3259B ESL W HISTORY B 
ENG1259B ENG 2B TRANS MTH2327A MATH MDL APPL A SST3359A ESL US HISTA 
ENG1264A ENG SOL 2A-BG MTH2327B MATH MDL APPL B SST3359B ESL US HIST B 
ENG1264B ENG SOL 2B-BG MTH2329A ESL MATH MDLS A 


25 


Appendix K 


9VZZ/ZZET GO’ < LZL'€ | LEZZ/GEETZ gO’ < T> L62Z/2ZETZ go’ < T> T6ZZ/LZEZ_—ss GO'< T> S9IPNIS [eIN0S 
v9OTZ/T8TZ Go’ < T> 6ZTZ/T6TZ Go’ < v89'z | v9TZ/O8Tz Go’ < T> 6LZTZ/98TZ = GO'< T> 99USIIS 
60TZ/SZTZ GO’ < T> 6OTZ/EETZ go’ < T> ZOTZ/ZTTZ gO’ < T> EZTZ/TETZ GO'< T> V1a/6ulpeay 
= TCTC/E8TS gO’ < 6027 ETTC/SG8TC gO’ < LECC OSTZ/9ST? g0°< T> 8LTZ/E8TS sOo°'< T> yew 
g o sueal d }sa] 4 suval d | ysa] 4 sueal d | jsa] 4 sued d 3S9} 4 | alqns pajsaL 
ax Salpnjs |e120S a9uUa19S Buipeay ule 
= 8 (paynuap]-s[as) yalqns s8yoeeL 
@-2 SMVL 
Bo 
£ = E6EE/ZBEE  —GO'< T> 66EE/ZIVE  GO'< | T> GOEE/G6EE GO'< GITT | Gzverezve sOo< T> Aydeib02a5 puom 
a g Q9EE/ECCE €00° 8c8'2 VOTE/LECE gO’ < vSc9 CECE/OCCE SO’ < T> 89EE/TVCE €00° 8ET'8 AJO\SIH PLOM 
= ‘oa 8LSE/VCSE gO’ < OOZ'T 9CSE/TESE gO’ < T> OE9E/0CSE vtO° SES v7 ST9OE/VVSE cc0° 680'7 Ajawoas 
o ~S 09SE/Z9VE Tv0O" EcO'E 96EE/COVE gO’ < ves T L6VE/SSVE SO’ < T> EGSE/ELVE SO’ < COT’? Auysiwayo 
= S  T99T/LZ9T 9Z0" LI8’€ | PGOT/OE9T Go’ < GlTZ@ | T8ST/ZZ9T GO'< T60'S | 999T/OV9T Zv0" 2662 | I Gun 
PY) © 909T/7O9T gO’ < T> 86ST/ECOT gO’ < STZ? 99ST/909T gO0°< 2Z0€'6 9TIT/9COT gO’< T> | Bunun 
1} = DOLT/ECLT g0°< vrS'~ SOLT/LELT gO’ < T> 9LOT/8TLT SO’< OLE? LO8T/LELT TOO’ 909°6 \| Buipeay 
nD  6S9T/L99T Go’ < T> 6L9T/G89T Go’ < T> €G9T/ZZ9T GO'< G8Z'T | 889T/O69T  SO'< T> | Guipeay 
. = 6TSE/VTSE SO’ < T> O8SE/8ESE g0°< €69'T OLSE/LTSE €0° c6S'€ CO9E/B7SE SO’ < OcLl'c ABoo1g 
2 5 _gsse/so9e —G0"< T> E6VE/LEQE  GO'< 8ZS°0T | ZO9E/66GE GO'< T> 8VGE/TS9E GO'< ese'z | | eaqably 
e & sueoW d 3s9} 4 sueoIN d 1Ss9} 4 SUedIN d 389} 4 sueoW d 3s9} 4 | Walqns pajsaL 
= = salpnys [els0S 39U919S Bulpeay yew 
ead (payinuap]-3[aS) 9alqns sayoeaL 
38 @S1N0Q-}0-puz YVVLS 
le 
il 5 6LEE/VEEE SO’ < 0SO'T TEVE/IEEE g0°< cOE'T LVCE/ETEE SO’ < 218°8 CECE/IEEE SO’ < S206 S9IPNIS [elI90S 
Os  Tevreiosve so'< T> BLGE/ZSVE GO'< ZvGT | ZZee/6zvEe GO'< Tl6€ | LOEE/ZGvE GO < TOT'y | BDUBIDS 
2 e 6OST/STST GO’ < T> vIST/VTST Go’ < T> ZTST/ZTST Go’ < T> QTST/ZTST  GO'< T> V1a/6ulpeay 
ms : 6TST/VCST g0°< T> vOST/7CST gO0°< 9110 STST/VCST SO°< vso'v CVST/VCST S00° 829°9 yew 
a g suealWN d 189} 4 sueolWN d 1Ss9] 4 sueoIN d 1sa} 4 suvalN d 189} 4 | Walqns pajsaL 
saipnys jel20Ss a0ua19S BHuilpeay ule 
(paynuap]-s[as) }alqns s9yoeaL 


YVVLsS 


26 


Appendix K (continued) 


27 


V'GL/II'BL SO’< cS0'9 LYVLI8'8L gO’ < LEEV OLLIL'BL SO’ < T> VELi6 BL gO’ < c6Ll ‘2 S9IPNIS Jel90S 
w SLLIT SL v00" OvL'2 T6L/7'SZ €Z0° LE0'V O'T8/T'SZ TOO’ gg9'9z | 8 6Z/S'SZ TOO" GLO'ZT | BDUBIDS 
LLLIG VL VIO" v88'r GOL/I6'vZ GO’ < TL2'9 ZLIB VL GO’ < 62T'v | 9VL/IO'SL GO’ < [> yyeW 
98Z/T'9L TOO" LOL‘LT 6B EL/T'OL gO’ < c9ov'cT 8°8Z/0'92 TOO" v6c V9 T9LIZ 92 gO’ < T> V1a/6uipesy 
sueal d }sa} 4 suval d jsa] 4 sueal d jsa] 4 sueol d 3Ss9] 4 yoalqns pajsaL 
Salpnys jel20Ss aoual9S BHulpeay ule 


(paynuap]-s[as) }alqns s9yoeeL 
sapeid asinog 


T'SG/8°6S So’ < gez'e | 6'9S/T09 gO'< e9e'T | €°99/5°6S TOO’ 60S'02 | 6'2S/9'09 gO’ < OEe'T | SSaubOld Auea,r 
9EL/BEL gO’< T> 8EL/6EL go’< T> TVL/LEL v00° 96e°2 Evliovl gO0'< 9¢29'T Airset 
sueal d }sa] 4 suval d jsa] 4 sueal d jsa] 4 sueol d 3S9] 4 yoalqns pajsaL 
SaIpnys jel20Ss a9ua19S Hulpeay ule 
(paynuap]-s[aS) }9alqns sayoeaL 
SVd141L 
9'0E/0'TE gO’ < T> 9 0E/T TE i 6 TE/T TE t 6° CE/C TE i abenbuey 
GCEIV'EE re) T> ECEIV EE ; CVEIV'EES i LECIE ECE BIUSIIS [eBl90S 
T 8E/T TV SO’ < c60°9 | 2 Ov/O' TV r VTv/C TV 8 Tr/0' TV f BIUSIOS 
8 lev le SO'< T> OLe/v Le ; Cc 82/V' LC Q 8 L2/9' Le : Bulpeay 
6 Tr/V'EV SO’< ECG? | VCcv/9 EV ; QEV/BEV O'E/L EV ; yyew 
SUeaIN d 1S} 4 SUedIN d 1S9} 4 SuedIN d 1S$9} 4 SUROIN d 3sa} 4 «| WOalqns palsaL 
SdIPN}s [elN0S a9uaI9S BHuilpeay yuleI 


(payinuap]-3[aS) }alqns sayoeaL 
OT psojuejys 


Results of Statistical Analyses of Student Performance Data 
Assessment Type, Subject Tested, and Teacher Subject Area 


